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Humans traditionally experience a vigilance decrement over extended periods of time on reliable systems. 
One possible solution to aiding operators in monitoring is to use polar-star displays that will show 
deviations from normal in a more salient manner. The primary objectives of this experiment were to 
determine if polar-star displays aid in monitoring and preliminary diagnosis of the aircraft state. This 
experiment indicated that the polar-star display does indeed aid operators in detecting and diagnosing 
system events. Subjects were able to notice system events earlier and they subjectively reported the polar- 
star display helped them in monitoring, noticing an event, and diagnosing an event. Therefore, these results 
indicate that the polar-star display used for monitoring and preliminary diagnosis improves performance in 
these areas for system related events. 


INTRODUCTION 

Humans traditionally experience a vigilance decrement 
over extended periods of time on reliable systems (Wickens & 
Hollands, 2000). As the automation of systems increases, the 
requirement for this type of monitoring necessitates the display 
of information to the human operator in such a way that he can 
notice pertinent deviations in a timely manner. One possible 
solution to this problem is to use polar-star displays that will 
show deviations from normal in a more salient manner. 

A polar-star display essentially is a polygon (Danchak, 
1981). When the polygon is regular, all the parameters 
represented by the polygon are at or near their expected. A 
deviation of a vertex indicates a non-normal parameter. For the 
polar-star displays used in this experiment, each vertex could 
represent a single parameter ( e.g ., indicated airspeed) or 
collection of parameters for a system ( e.g ., hydraulic system 
pressure and temperature). 

Previous Research 

A polar-star display was chosen primarily because research 
indicates that using geometric shapes to assist the operator 
integrate information improves performance (Barnett & 
Wickens, 1988; Cooper, 1980; Dinadis & Vicente, 1999; Jacob, 
1978; Jacob & Egeth, 1976; Jones, Wickens, & Deutsch, 1990; 
Wickens & Hollands, 2000). Trujillo et al., reported research 
that established the optimal parameters for each vertex (Trujillo 
& Schutte, 1999), movement of the display vertices (Bartolone 
& Trujillo, 2002; Trujillo, 2002), and where the display should 
be located (Bartolone & Trujillo, 2002). 

In the context of the flight deck, this display is envisioned 
to be used primarily for monitoring and preliminary diagnosis. 
The reasons for this are numerous. First, the information 
provided on the display qualitatively notifies an operator when 
a parameter is deviating from normal but it does not 
quantitatively tell him by how much. Furthermore, in some 
cases the display does not tell the operator exactly which 
parameter is deviating. The operator will need to refer to more 
detailed aircraft instruments to locate this information. 

Second, the display provides generalities and not specifics; 
therefore, it cannot be used for detailed diagnosis (Barnett & 
Wickens, 1988). Since the display only provides an overview, 


it can be used for general failure detection for those parameters 
included in the polar-star display (Buttigieg & Sanderson, 

1991; Cooper, 1980). 

Lastly, other formats, such as the deviation bar graph, are 
better for more detailed status information (Abbott, 1989, 

1990) but the polar-star display may perform better when 
viewed at a glance (Mahaffey, Horst, & Munson, 1986). In this 
case, operators are only monitoring and making preliminary 
diagnosis with the display, not controlling the system. 

Therefore, for this study, two polar-star displays were used 
that indicated mission parameters and system parameters 
(Trujillo & Schutte, 1999). The displays are shown in Figure 1. 
The polar star on the left shows mission parameters while the 
polar star on the right shows system parameters. The dotted 
circle indicates the optimal placement of the vertex given the 
current aircraft configuration. Even though other research has 
indicated that directional information is not needed in detecting 
the movement (Harvey & Michon, 1974), the movement of the 
vertices was continuous and directional because of preferences 
pilots expressed when viewing a polar-star display (Bartolone 
& Trujillo, 2002; Trujillo, 2002). Lastly, the polar-star 
displays were placed in the traditional central area of a glass- 
cockpit set-up; i.e., near the alerting screen (Bartolone & 
Trujillo, 2002). 


Objectives 


The objectives of this experiment were three-fold. First, 
determine if the polar-star displays aid in monitoring the 



HDG=heading 
CRS=course 
FLS=fuel score 
DST=distance 
IAS=indicated airspeed 
VS=vertical speed 


CS- - - -AIR 



AIR=a/c, pressurization 

HYD=hydraulics 

ELC=electrical 

AVN=avionics 

FL=fuel 

ENG=engine 

CS=control surfaces 


Figure 1. Experiment Polar-Star Displays 


aircraft state. The polar-star displays will concentrate aircraft 
state information into a central location. This should assist 
subjects in monitoring the aircraft. With this centralization, it 
is expected that subjects will generally prefer the new display 
over the baseline condition. Besides having the aircraft 
information in a central location, the polar-star display is 
expected to aid subjects in detecting parameters deviating from 
normal because of the general design of the display. 

Second, see if the display assists with the preliminary 
diagnosis of aircraft state. Once the subject detects a deviation, 
the display should inform him of the general nature of the 
problem. This information will hopefully narrow down the 
subject’s search for information to diagnose the abnormality. 

Third, ascertain whether the groupings of aircraft state 
information were appropriate. Whether the groupings of 
parameters were logical will become readily apparent from the 
ease of monitoring and diagnosing of events. This will be 
reflected in the time it takes subjects to notice an event, and 
situation awareness (SA) and workload levels when using the 
new displays. 

EXPERIMENT DESIGN 
Experimental variables 

There were two experimental variables: display 
configuration and event type. Each subject ran half of the data 
runs with the baseline configuration and half of the data runs 
with the baseline configuration plus the polar-star displays. 
Furthermore, half of the non-normal events were events that 
affected mission parameters and half of the non-normal events 
affected the system parameters. 

Display configuration. Each subject saw the two display 
configurations: baseline and baseline plus the polar-star 
displays. The polar-star displays were located to the right of 
the system displays. The order that the subjects saw the 
displays was counterbalanced. 

Event type. The event types were grouped into either non- 
normal events affecting mission parameters ( e.g ., altitude bust) 
or non-normal events affecting system parameters ( i.e ., high 
cabin altitude). Only one non-normal event would occur 
during a data run. Each subject had a randomized order of 
events; although in each display configuration, half of the runs 
consisted of non-normal events affecting mission parameters 
and half of the runs consisted of non-normal events affecting 
system parameters. 

Subjects 

Twelve people participated as subjects. All subjects were 
currently certificated transport pilots who flew glass-cockpit 
aircraft. The average age of the pilots was 45 years old with an 
average piloting experience for passenger revenue of 13 years. 

Test design 

This experiment was conducted in the Intermediate Design 
and Evaluation Simulation Lab at the NASA Langley Research 
Center. The baseline configuration used in this low-to-medium 


fidelity single-pilot simulator was that of a Citation Ten (a 
high-end business jet with 2 engines) (Cessna, 1996; 
FlightSafety International, 1999) with the Floneywell Primus 
2000 avionics suite (an advanced glass cockpit) (Honeywell, 
1997). 

As mentioned earlier, each subject saw the baseline 
configuration and the polar-star display with non-normal events 
that encompassed mission parameters or system parameters. 
With twelve data runs, this means that there were three 
repetitions for each display configuration and fault type 

Dependent measures 

Objective measures. Besides the aircraft state variable 
information recorded (e.g., altitude and indicated airspeed), 
additional objective measures were taken. These included the 
point at which a subject first noticed a non-normal event 
occurring and when he felt he could diagnose it. This was 
accomplished by having the subject push buttons labeled 
“PROBLEM DETECTED” and “PROBLEM DIAGNOSIS” 
located above the navigation display when he thought he first 
detected an event and then when he knew what the event was. 
This time was recorded by the simulation. 

Furthermore, his diagnosis of the non-normal event was 
also recorded after each data run. The subjects’ diagnoses were 
made only to the level of “altitude bust” or “hydraulic 
quantity.” 

Subjective measures. After each data run, the subjects 
answered SA questions about current aircraft state from 
memory (i.e., amount of fuel left). The SA questions were 
modeled after Endsley’s SAGAT method with the primary 
difference being that the questions were only asked at the end 
of each data mn (Endsley, 1995a, 1995b, 2000). Subjects also 
completed a NASA-TLX for workload (Byers, Bittner, & Hill, 
1989; Hart & Staveland, 1988). After all the simulation runs, 
each subject completed a final questionnaire that asked for his 
preferences on the displays just seen. 

Procedure 

When a subject first arrived, he was given an overview of 
this experiment. He then received instructions on how the 
simulation worked and the new displays. He also was briefed 
on the route of flight and given a map and flight plan that he 
was to follow. Lastly, the subject received an overview of the 
questions he would have to answer after each data run. 

After this briefing, the subject went to the simulator where 
he was able to familiarize himself with the simulation of the 
aircraft. Before any data runs after a display configuration 
change, the subjects had two practice runs that behaved the 
same as the data runs. After the two practice runs, the six data 
inns with the first display condition were completed. 

During the initial part of the data run, subjects were able to 
familiarize themselves with the aircraft location in space and 
the aircraft setup. Two to ten minutes into a run an event 
occurred. When the subject noticed an event, he recorded this 
with a push of the “PROBLEM DETECTED” button. Once 
the subject felt he could make a preliminary diagnosis of the 
event, the simulation stopped and the subject was asked to 



answer SA questions, make a preliminary diagnosis, indicate 
which screens were the most beneficial to noticing and 
diagnosing the event, and give a workload rating. When the 
data runs for the first display configuration were finished, the 
display configuration was changed and the subject had two 
practice runs with the new configuration before the six data 
runs began. 

Finally, at the end of all the data runs, subjects completed a 
questionnaire asking them about their display preferences. Any 
other comments were also recorded at this time. 

Data analysis 

Data were analyzed using SPSS 8 (SPSS Inc., 2002). The 
event detection time, event diagnosis time, SA, and NASA- 
TLX workload were analyzed using a repeated measures test 
with diagnosis accuracy as a covariate. 

The event detection time and event diagnosis time were 
computed as the difference between when the subject pushed 
the “PROBLEM DETECTED” and “PROBLEM 
DIAGNOSIS” buttons and the start of the event. The SA 
measure was a summation of each question the subject 
answered. The terms in the summation were the absolute value 
of the subject’s answer minus the correct answer divided by the 
overall standard deviation for that variable. The NASA-TLX 
workload measures were calculated by doing a simple average 
of the six measures ( i.e ., mental, physical, and temporal 
demand; performance; effort; and frustration level). 

The repeated measures for these analyses were the 
repetitions. In those cases where the repetitions were not 
significant, a univariate analysis was done. Order effects were 
checked for but were only present for workload. The end 
questionnaire data were analyzed using a T-Test. In all cases, 
significance was set at p<0.05. 

Diagnosis accuracy was broken down into 4 categories. 

The highest category was “Totally Correct,” which meant that 
the subject correctly identified the non-normal event. 
“Subsystem Correct” indicated that the subject knew the 
subsystem ( e.g ., electrical system) but not the exact non-normal 
event (e.g., battery temperature). The “Mission/System 
Correct” category meant that the subject understood that the 
non-normal event affected either the mission or system. Lastly, 
“Totally Incorrect” meant that the subject did not know 
whether the non-normal event affected the mission or system. 

RESULTS 


Event Detection Time 

For event detection time, diagnosis accuracy (-Ei,i 2 8 = 8.88, 
p<0.01), event type (7 7 i i 2 8=6.15, p<0.02), and event type by 
display configuration (F 1;128 =6.96, p<0.01) were all significant. 
As diagnosis accuracy improved, event detection time 
increased (Figure 2). Subjects detected mission event types a 
minute earlier, on average, than they detected system events 
(99 s and 160 s respectively). Subjects noticed system events 
faster with the polar-star display than without it, but they 
detected mission events slower with the polar-star display 
(Figure 3). Lastly, subjects reported that it was easier to 


monitor and to notice events using the polar-star display with 
the greatest improvements for monitoring system status and 
noticing system events (Table 1). 

Event Diagnosis Time 

Diagnosis accuracy (Fu 28 =13.75, p<0.01) and event type 
(^j,i 2 8~3.97, p<0.05) were significant for event diagnosis time. 
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Figure 2. Event Time vs Diagnosis Accuracy 


250 



Baseline Polar Star 

Display 


Figure 3. Event Detection Time vs Display & Event Type 


Table 1 - End Questionnaire Subjective Ratings of Polar-Star 
Display Compared to Baseline Display 


Monitoring 

From 

Baseline 

Standard 

Error 

Mission Status 

-15.12 

5.48 

System Status 

-37.50 

3.09 

Noticing 



Mission Event 

-7.50 

4.92 

System Event 

-38.10 

3.04 

Determining 
Status of 



Mission 

-11.55 

5.32 

System 

-37.02 

3.30 

Workload of 
Monitoring 



Mission Status 

-13.21 

2.76 

System Status 

-30.71 

5.56 


Note: 100 point scale (-50 to 50) with 0=baseline 
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As with event detection time, as event diagnosis time 
increased, diagnosis accuracy also increased (Figure 2). 

Subjects also diagnosed mission events about a minute before 
they were able to diagnose system events (124 s and 185 s 
respectively). 

Finally, subjects reported that it was easier to determine the 
status of events using the polar-star display again with the 
greatest improvement for system related parameters (Table 1). 

Workload 

interestingly, the order subjects saw the displays was the 
only statistically significant factor (F l ,142=10.65, p<0.01) in the 
workload analysis. Subjects who saw the baseline condition 
and then the polar-star display had lower workload ratings by 
approximately 5% (28 and 34 respectively out of a 100 point 
scale where 0 represents lowest workload) than subjects who 
saw the polar-star display and then the baseline display. 

In the end questionnaire, subjects reported that the 
workload was lower with the polar-star display for monitoring 
mission status and system status (Table 1). As with event 
detection time and event diagnosis time, the workload of 
monitoring system status had a greater improvement than 
mission status. 

Situation Awareness 

For SA, repetitions ( F iA2 =1.51 , p<0.01) and repetitions by 
event type (F 3 , 42 = 3 . 16 , p<0.04) were significant in the repeated 
measures test. From looking at the data, the first repetition was 
driving these results. Once it was removed, repetition was 
barely significant (7 7 2 ,43=3.24, p<0.05) and repetitions by event 
type was not significant. Therefore, a univariate analysis was 
done for the repetitions after repetition 1 . 

For the univariate analysis, event type was statistically 
significant (iq, 88=4.97, p<0.03). The system event type SA was 
higher than for the mission event type SA by approximately 8 % 
(92 and 85 respectively on a 100 point scale where 100 is the 
maximum SA); although for both types of events, the SA was 
very high. 

Diagnosis Accuracy 

Event type was significant for diagnosis accuracy 
f 7 U28=14.44, p<0.01). Diagnosis accuracy was higher with 
system events than for mission events (Figure 4). 

DISCUSSION 

In general, subjects were able to detect system events 
earlier with the polar-star display. Subjects reported that the 
polar-star display did assist them in monitoring, noticing an 
event, and status determination. Again, the improvement was 
for system events rather than for mission events. The polar-star 
display format for system parameters seems to have increased 
perceptual sensitivity due to its movement and pattern 
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Figure 4. Diagnosis Accuracy vs Event Type 

recognition attributes (Donald, 2001; Sanders & McCormick, 
1987). The movement and pattern recognition aspects of the 
polar-star display may have also aided subjects in making 
predictions which would obviously affect their detection and 
diagnosing times (Hawkins, 1993). 

Not surprisingly, diagnosis accuracy increased with 
increasing time. The more time subjects searched for 
corroborating evidence, the higher their accuracy. Subjects 
were also more apt to make a diagnosis earlier for mission 
events although their diagnosis accuracy was only better for 
system events. This is most likely because system events 
generally have alerts associated with them that essentially told 
the subjects the exact nature of the event. 

Workload measures improved for the polar-star display by 
the end questionnaire but the NASA-TLX measurements after 
each data run did not indicate an improvement using the polar- 
star display. This overall decrease in workload when using the 
polar-star display, especially for system parameters, probably 
reflects the fewer cognitive processes subjects went through 
when using the display to determine if anything was deviating 
from normal. A quick glance sufficed rather than 
comprehending a quantitative parameter value (Sanders & 
McCormick, 1987). Workload was also less for subjects who 
saw the baseline display and then the polar-star display than for 
subjects who saw the polar-star display before the baseline 
display. The lower workload for subjects starting with the 
baseline condition was most likely a result of them starting 
with a known display setup and then adding the new display. 
The initial baseline condition gave the subjects time to become 
comfortable with the new situation without adding the extra 
stressor of learning to use a new display. 

Also, SA improved for system events. For both the 
workload and SA measures, the differences were statistically 
significant, but numerically, the differences are small. At the 
very least, the new display does not increase workload or 
decrease SA. 

One benefit the polar-star display has is to collocate 
information onto one display. For system parameters, the polar 
star accomplishes this since system information is otherwise 
spread out on overhead panels and multifunction displays. 
Mission information, consisting mainly of aircraft state 
parameters, is located centrally on the primary flight display 
and navigation display. Therefore, this information is already 
aggregated for the operator. 





The polar-star display for system information also does not 
require the operator to delve into menus to reach the 
information. Therefore, the system information using the 
polar-star display becomes readily available like the mission 
information already on the primary flight and navigation 
displays. 

Lastly, since so much of the system information is layered 
in multifunction displays and the systems are highly 
dependable, most operators rely heavily on the alerting system 
to notify them of deviations. The polar-star display, being in 
the scan pattern, alerts operators of deviations before an alert 
range is reached. 

Therefore, these results indicate that the polar-star display 
used for monitoring and preliminary diagnosis improves 
performance in these areas for system related events. 

CONCLUSIONS 

As the automation of systems increase, the requirement for 
monitoring necessitates the display of information to the human 
operator in such a way that he can notice pertinent deviations in 
a timely manner. One possible solution to this problem is to 
use polar-star displays that will show deviations from normal in 
a more salient manner. 

The greatest benefit the polar-star display appears to derive 
from the fact that it collocates system information onto a single 
display. Diagnosis of system events is typically easier than for 
mission events because the vast majority of system events have 
alerts associated with them. But monitoring and detecting a 
system event is more difficult due to this information being 
spread out and layered. The ability of the polar-star display to 
aggregate this information seems to be akin to the primary 
flight display and the navigation display for mission 
information. Consequently, since mission information is 
already collocated on the primary and navigation displays and 
readily available within pilots’ scan patterns, the mission polar 
star did not show the benefits that the system polar-star display 
did. 

Therefore, this current experiment indicated that the polar- 
star display does aid operators in detecting and diagnosing 
system events. Subjects were able to notice system events 
earlier and they subjectively reported the polar-star display 
helped them in monitoring, noticing an event, and diagnosing 
an event. Lastly, they reported a slightly lower workload with 
the polar-star display. 
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